\documentclass[12pt]{article}
\usepackage{titlesec}
\usepackage{graphicx,setspace,hyperref,amsmath,amsfonts,times,multirow,ccaption,tabulary,verbatim,booktabs,mdwlist}
\usepackage[top=1in, bottom=1in, left=1in, right=1in]{geometry}
\setlength{\marginparwidth}{.5in}
\usepackage{natbib}
\bibpunct{(}{)}{;}{a}{}{,} %set in-line reference punctuation
\setlength{\bibsep}{0.1in} %set spacing between references
\setlength{\bibhang}{.5in} %set hanging indent for references
\setlength{\parindent}{0.5in}
\renewcommand\bibsection{\section*{\refname}} %Changes "Bibliography" to "References"
\expandafter\def\expandafter\quote\expandafter{\quote\singlespacing} %single space quote environment
\newcommand{\tablenote}[1]{\singlespacing\footnotesize Note: #1}
\newcommand{\tabref}[1]{\begin{center}\textbf{Table \ref{#1} Here}\end{center}}

\usepackage[T1]{fontenc}
\usepackage{lmodern}
\hypersetup{
    bookmarks=true,         % show bookmarks bar?
    unicode=false,          % non-Latin characters in Acrobat’s bookmarks
    pdftoolbar=true,        % show Acrobat’s toolbar?
    pdfmenubar=true,        % show Acrobat’s menu?
    pdffitwindow=false,     % window fit to page when opened
    pdfstartview={FitH},    % fits the width of the page to the window
    pdftitle={Cognitive Style and the Survey Response},    % title
    pdfauthor={Thomas J. Leeper},     % author
    pdfsubject={public opinion},   % subject of the document
    pdfkeywords={public opinion} {need to evaluate} {opinion formation} {survey}, % list of keywords
    pdfnewwindow=true,      % links in new window
    pdfborder={0 0 0}
}

\title{Cognitive Style and the Survey Response}
\author{Thomas J. Leeper}

\begin{document}

\maketitle

\noindent Running Head: Cognitive Style and the Survey Response

\vspace{1em}

\noindent Author information:\\Thomas J. Leeper\\Department of Political Science and Government\\Aarhus University\\Bartholins Alle 7\\8000 Aarhus C\\Denmark\\+45 8716 5592\\thosjleeper@gmail.com

\clearpage

\noindent Author note:\\Thomas J. Leeper is a postdoctoral researcher in the Department of Political Science and Government, Aarhus University, Aarhus, Denmark.

\vspace{1em}

\noindent Acknowledgements:\\The author wishes to thank the editors and anonymous reviewers for their helpful feedback on earlier drafts of this paper. Full replication data and code for all analyses is available at the Harvard IQSS Dataverse: doi:10.7910/DVN/V9ROA.


\clearpage


{\abstract A lack of well-defined, valenced opinions is seen as a violation of the core expectations for citizens laid out in democratic theory. Yet evidence suggests individuals vary in chronic evaluative tendencies, with some processing information in an online fashion, spontaneously becoming opinionated about whatever they encounter and recalling summary evaluations easily. Individuals lower in evaluative tendencies do not form attitudes in this fashion and instead must construct and report attitudes based upon recalled considerations. The implications of these individual differences for survey practice are largely unexamined. The present research shows, across multiple surveys and question formats, that individuals low in need to evaluate (NE) offer greater numbers of non-informative responses to opinion questions than those higher in NE, an effect not attributable to interest, knowledge, or lack of cognitive effort. And, when they do supply answers to opinion questions, they report attitudes less extreme than individuals higher in NE. The results imply that opinion surveys may systematically misrepresent the variability of citizens' beliefs and the extremity of aggregate public opinion by relegating some of those low in NE to non-informative response options.}

\clearpage

\doublespacing

How and why citizens seem to lack opinions on major issues, including those that are contentious and politicized, has puzzled public opinion researchers for decades \citep[see, for example,][]{Converse1964, Neuman1986}. Most of the answers to these questions have related to lack of sophistication or a deficit of interest, either in the survey interview or in politics generally. A lack of well-defined, valenced opinions is seen as a violation of the core expectations for citizens laid out in democratic theory \citep{Dahl1971}. Yet little attention has been paid to the reality that some individuals are predisposed to form opinions ``online'' while others do not readily form such evaluations. Indeed, evidence suggests evaluative individuals update opinions automatically when processing information in their environment and recall those summary evaluations easily \citep{LodgeMcGrawStroh1989, DruckmanLupia2000}. Individuals lower in evaluative tendencies do not form attitudes in this fashion, storing information without the construction of a running tally opinion, and instead must construct opinions based upon recalled considerations. What effect does this have on survey responses? In particular, how does cognitive style impact response patterns on opinion questions?

The obvious expectation is that evaluative individuals, being more prone to hold online opinions, should be more likely to supply valenced opinion responses and more extreme responses, while those who form memory-based opinions are more likely to supply non-informative responses. As such, opinions recorded in survey responses may reflect an unintended interaction between cognitive style and survey procedure. Yet surprisingly little attention has been paid to how cognitive style might shape survey responding. %\footnote{Of course, the present research is not novel in expressing concern for non-informative survey responses more generally. \citet{Berinsky1999}, for example, was concerned with how selection into DK answers silences the political views of particular groups \citep[see also][]{Berinsky2002, Berinsky2004}. He notes that the tendency to express attitudes necessarily translates into greater political power; unexpressed viewpoints cannot be represented. \citet{Krosnicketal2002} also examined how question format --- specifically the use of explicit ``no opinion'' response options --- affects survey responding. While some have argued that providing a ``no opinion'' option allows those with genuinely no opinion to avoid social pressure to respond with an opinion, their research shows that providing ``no opinion'' response options does not improve data quality and invites satisficing behavior. A recent paper by \citet{SturgisRobertsSmith2014} also shows that, when prompted, 53\% to 84\% of respondents selecting neutral middle category responses admit to choosing that option to hide a DK response. Expressions of non-informative responses on surveys therefore do little to improve data quality and may have deleterious political consequences when findings from public opinion research is used for policymaking.}
This matters because public opinion research often denigrates those who fail to express valenced attitudes, when those individuals may simply lack the ``online'' attitude summaries necessary for quick survey responding. This paper asks whether there is a linkage between cognitive style (i.e., evaluative disposition) and patterns of response to opinion questions. Need to evaluate (NE) \citep{JarvisPetty1996}, a standard measure of evaluative cognitive style, is found to be a strong predictor of non-informative responses in American National Election Studies (ANES) surveys from 2000, 2004, 2008, and 2008-2009, independent of variations in political interest, political information, need for cognition, and other stable demographics. The paper then examines the implications of that pattern for the extremity of reported opinions and whether changes to survey procedures aimed at accommodating memory-based processing might reduce non-informative responding.

\section*{Evaluation and Attitudes}

Surveys consistently suggest that some people have no opinion on political issues, including those that are high-profile, contentious, and widely discussed in media. Two common interpretations of these survey responses focus on political interest and survey engagement. Either non-informative responses reflect ``nonattitudes'' \citep{Converse1964, NorpothLodge1985} and the electorate's political disinterest \citep[see, for example,][]{Neuman1986}, or they reflect survey satisficing, where respondents fail to engage sufficient effort in the survey interview \citep{Krosnick1991, KrosnickNarayanSmith1996, TourangeauRipsRasinski2000}. While political interest and survey satisficing likely explain some of the variation in survey responding, an unexamined possibility is that survey responding also reflects individual differences in cognitive style.

Research suggests that individuals vary on (at least) two dimensions of cognitive style, which are commonly defined and operationalized as Need for Cognition (NC) \citep{CacioppoPetty1982} and Need to Evaluate (NE) \citep{JarvisPetty1996}. NC measures individuals' disposition to think about information in their environment. NE measures individuals' tendencies to {\em evaluate} that information. High-NE individuals engage ``online'' processing \citep{LodgeMcGrawStroh1989}, storing readily accessible summary evaluations of (political) objects in memory. Low-NE individuals tend to be memory-based processors, learning about their environment but not forming such summary evaluations. Despite growing use of NE as a predictor of vote choice and candidate evaluation \citep{Bizeretal2004, Holbrook2006}, as a moderator of framing effects \citep{ChongDruckman2010, DruckmanLeeper2012a}, and as a proxy for motivated reasoning \citep{Nir2011}, the only published account linking NE and survey responding comes from a small student sample and shows that high-NE students are less likely to supply ``don't know'' (DK) responses, but with no other effects on survey response patterns \citep[see][]{JarvisPetty1996}. Another unpublished paper, using the ANES 1998 Pilot Study, finds only minimal evidence for an effect of NE on DK responding \citep{Bizeretal2000}. \citet{Federico2004} finds that NE and NC interact to affect attitude extremity but does not examine other response patterns.

How does NE affect responses to opinion questions? High-NE individuals can simply retrieve online opinion summaries from memory while low-NE individuals have no such summaries to retrieve, instead needing time to construct opinions from stored considerations. As such, we should expect higher NE to be associated with fewer non-informative responses to opinion questions. By contrast, low-NE individuals should be more prone to providing non-informative responses to opinion questions. The latter group may also be less extreme when they do respond due to the short time they have to assemble opinions from stored considerations during the course of a survey interview.

The remainder of this paper first tests for the expected relationship between NE and non-informative survey responses and then explores whether NE is related to opinion extremity. The results are then discussed in light of additional evidence that rates of non-informative responses to opinion questions are dramatically reduced under ``stop-and-think'' question procedures, precisely the conditions needed to allow low-NE individuals to construct memory-based responses.


\section*{Data and Methods}

Data are drawn from the ANES 2008-2009 Online Panel and the ANES 2000, 2004, and 2008 election surveys. This combination of data is used because the Online Panel and election surveys use two different response formats on opinion questions. The 2008-2009 ANES Online Panel measured opinions on eight different policies at each of two points in time (Waves 1 and 10).\footnote{The sample for Wave 1 (Jan 2008) involved 1,623 respondents and Wave 2 (October 2008) involved 2,628 respondents, recruited from a landline RDD frame. AAPOR RR1 was 18\% for Wave 1 and 16\% for Wave 2.} All questions were asked with identical wordings at both panel waves, involved identical response options, and used a branching format. Respondents were asked to select among `favor,' `oppose,' and `neither favor nor oppose' response options. Afterward, those respondents selecting `favor' or `oppose' were asked about the degree of their support. In coding non-informative responses, respondents who initially provided no answer or selected `neither favor nor oppose' were coded 1 as providing a non-informative response and 0 otherwise.\footnote{Except in the case of extremely unusual, unimportant, or unfamiliar issues where one might be informed but undecided, `neither favor nor oppose' is semantically equivalent to offering a DK or `no opinion' response \citep{Bishop1987}. Indeed, research by \citet{SturgisRobertsSmith2014} shows that once prompted half or more of middle-category respondents admit to choosing the non-informative cateogry in order to avoid having to say ``don't know.''} Online Appendix A includes the numbers and percentage of respondents providing each type of response, along with exact question wordings for each question.\footnote{Because few respondents did not answer the question, the results are identical when only `neither favor nor oppose' responses are examined.}

NE was measured with two questions (which have been standard on the ANES since 2000; see \citealt{Bizeretal2000}). The first reads ``Some people have opinions about almost everything; other people have opinions about just some things; and still other people have very few opinions.  What about you?  Would you say you have opinions about almost everything, about many things, about some things, or about very few things?'' The second reads ``Compared to the average person, do you have fewer opinions about whether things are good or bad, about the same number of opinions, or more opinions?'' and includes a follow-up asking whether the respondent has a lot [more/fewer] or just somewhat [more/fewer] opinions. Throughout the analysis, these two questions are averaged and rescaled to range from 0 to 1.\footnote{For the first NE question, three respondents provided no answer. For the second question, six respondents provided no answer. Consequently missingness on the key independent variable should be relatively unproblematic. The mean and SD of the resulting scale were 0.58 (0.20).} The main analysis regresses the proportion of non-informative responses across questions on NE and a number of covariates.

The ANES surveys from 2000--2008\footnote{The 2000 survey of 1807 respondents was collected through a mix of in-person and telephone interviews between September and December 2000. In-person respondents were drawn from an area probability sampling frame and telephone respondents from a landline RDD frame. The response rate was 61.2\% for the pre-election wave and 86\% for the post-election wave. The 2004 sample involved 1211 respondents, was drawn via area probability sampling, and used in-person interviewing. The pre-election response rate was 66.1\% and the post-election rate was 88.0\%. The 2008 survey of 2,323 respondents was an area probability sample --- oversampling Latinos and African Americans -- with a RR1 of 59.5\% for the pre-election wave and 53.9\% for the post-election wave.} offered three standard opinion questions --- about government spending, jobs, and aid to blacks --- that use an unlabeled middle response category along with an explicit ``haven't thought about this much'' option. No answer, a DK, or ``haven't thought about this much'' were considered non-informative responses (and coded 1) while all other responses (including those providing a middle category response) were coded 0. NE was measured and coded in these surveys in the same way as in the Online Panel.\footnote{In the 2000 survey, four respondents provided no answer to the first NE question and thirty-one provided no answer to the second question. In 2004, these numbers were one and zero, respectively. In 2008, the numbers were two and seventeen, respectively. Means and SDs by year were 0.56 (0.23), 0.58 (0.21), and 0.55 (0.22), respectively.} To analyze the relationship between cognitive style and non-informative responding, data from 2000, 2004, and 2008 was pooled and non-informative responding on each question was regressed on NE and the same set of covariates used in the Online Panel analysis. Year fixed effects were also included (with 2000 as baseline).

\subsection*{Effects of NE on Non-informative Responding}


\tabref{tab:props2008-2009}

On the Online Panel, as many as 25\% of respondents offered non-informative responses, including on issues such as gay marriage, government-sponsored health care insurance, and a guest worker program for immigrants (see Table \ref{tab:props2008-2009}). Controlling for potentially confounding factors (e.g., that citizens have low political interest or knowledge) and demographics, the regression results in Table \ref{tab:full2008-2009} suggest that cognitive style has a consistent effect on individuals' survey responses. The first two columns show results of regressions of non-informative responses (as a proportion of the eight questions, separately for each of the two waves) on NE, NC, and many other covariates. As is clear, NE has a consistent negative effect on non-informative responding (i.e., those higher in NE are less likely to offer non-informative responses). The results in the third column pool the responses from both survey waves and again find a negative relationship between NE and non-informative responding. No other variable consistently explains these responses.\footnote{Additional robustness checks involved treating the responses to each survey question at both waves as a 16-wave panel and controlling for issue fixed-effects. The same results were found.} 

\tabref{tab:full2008-2009}

That NE but not NC is associated with non-informative responding suggests that such responses reflect deficits of evaluation not deficits of cognitive effort or engagement on the part of respondents. This is further substantiated by the results of a nonequivalent outcome test (available in Online Appendix B), which examined the effect of NE on DK responses to factual knowledge questions in order to test whether low-NE is associated with non-informative responding in general (and not just on opinion questions). These tests show, however, only modest differences in DK responses to factual questions attributable to NE. Their non-valenced attitude expressions seem to reflect something unique about how cognitive style interacts with opinion components of survey questionnaires.

\tabref{tab:anestsdesc}

On the ANES 2000, 2004, and 2008 surveys, fewer respondents offered non-informative responses to the selected opinion questions (see Table \ref{tab:anestsdesc}). Regressions of these responses on NE and other factors are shown in Table \ref{tab:fullts}. As is clear from the coefficients on NE, evaluative individuals were much less likely to provide a non-informative response on each of the three questions. While NC, political interest, and political knowledge sometimes shaped responding, these relationships were smaller and less consistent than the relationship between NE and non-informative responding. Thus, the results from the Online Panel are replicated and the results here suggest that the effect of NE may vary across question formats.

\tabref{tab:fullts}


\subsection*{Does NE also affect opinion extremity?}

The results thus far suggest that NE strongly relates to whether a respondent provides a non-informative answer to opinion questions. One obvious follow-up question is whether NE also affects other types of survey responding, such as the extremity of reported attitudes. As NE measures underlying evaluative tendencies, it is plausible that NE is also related to the extremity of opinion self-reports because high-NE individuals are more more likely to evaluate informational stimuli consistent with their priors and thus develop more extreme positions over-time \citep[see, for example,][]{Nir2011}. We might therefore expect that high NE is associated with more extreme responses.

To test for such a relationship, the extremity of attitude reports (from the same opinion questions already described) was regressed on the same set of covariates used in the previous analyses. On the 2000, 2004, and 2008 surveys, responses were ``folded'' so that middle-category responses were scored as 0 and the three response options on either side were coded 0.33, 0.67, and 1, respectively. On the Online Panel, the middle category (a non-informative option above) was excluded from the analysis and extremity of the remaining six response options were coded 0, 0.5, and 1, respectively. All ``don't know'' and non-answers were removed from all data.

Tables \ref{tab:extremity0809} and \ref{tab:extremityts} report the results for the Online Panel and the Time Series studies, respectively. In both cases, NE is associated with greater extremity of reported attitudes, among those reporting an attitude at all. Indeed, NE stands out as the largest single predictor of attitude extremity.\footnote{Additional analyses reported in Online Appendix C further replicate these findings for the spending/services question used on the 1998 ANES Pilot Study.}

\tabref{tab:extremity0809}

\tabref{tab:extremityts}

\section*{Addressing differences in survey responding}

Summarizing the results thus far, taking the results of these analyses together with the earlier findings, those high in NE are more likely to offer opinionated responses and when they do, those responses are more extreme than their low-NE peers. Low-NE individuals likely hold object-relevant beliefs in memory but simply lack the online summary necessary to quickly translate those beliefs into opinion responses, meaning surveys systematically mask the views of many citizens. Different question wordings and response options might help reveal those beliefs, e.g., removing explicit non-informative options may encourage low-NE respondents to provide a valenced response. But changes in response categories only affect response behavior on the margins without addressing the underlying differences in psychology that make it difficult for low-NE individuals to formulate opinions. It is possible that if individuals are given time and explicit instructions to recall issue-relevant considerations from memory before attempting to formulate an attitudinal response, they may be less likely to provide a valenced attitude expression.

\tabref{tab:zallerfeldman}

To shed some light on that possibility, I turn to an experiment on survey responding from from the 1987 ANES Pilot Study \citep{ZallerFeldman1992, FeldmanZaller1992}.\footnote{The sample of 457 respondents was drawn from a knowledge-stratified sample of respondents to the 1986 ANES time-series survey, with a response rate of 72\%. Telephone interviews were conducted between May and July 1987.} In one condition, respondents were asked to report opinions on the three questions used on the ANES 2000--2008 surveys, as normal. About one-quarter of responses to these questions are DKs. In the second ``stop-and-think'' condition, however, respondents were asked to first list considerations from memory then report their opinion. Only about three-percent of responses in this condition were DKs. The results are shown in Table \ref{tab:zallerfeldman}. Though the survey lacks a measure of cognitive style, a comparison of these two conditions provides a conservative test of how many more respondents are included in the informative response options under ``stop-and-think'' procedures. The ``stop-and-think'' procedures seem to provide more inclusive opinion estimates. 


\section*{Discussion}

This paper has shown that those low in NE are, under the conditions of standard survey practice, more likely to say that they do not know or have no opinion about a political issue when asked during a survey interview. The reason for this is a lack of habitual evaluation of stimuli, a simple individual difference. Such responses mean that their voices --- whatever they may know, think, or believe about political issues --- are hidden into the nonresponse categories of each question, discarded from aggregate estimates of public opinion. Evidence that NE is associated with more extreme responses raises additional questions about what public opinion estimates would look like if low-NE individuals were not excluded by way of DK and no answer options. Do they truly lack any issue-relevant beliefs? Or do their non-informative responses mask extreme or centrist beliefs? That high-NE individuals appear to have more extreme opinions means that aggregate opinion, including low-NE individuals may be more centrist than it appears. Yet we simply do not know without giving low-NE individuals time and opportunity to formulate opinions in a memory-based fashion. Invoking ``stop-and-think'' procedures, at least for those low in NE, may help to create more inclusive estimates of public opinion, but more work is needed to ensure that the survey interview works equitably for all respondents.



\clearpage
\singlespacing
\bibliographystyle{apsa-leeper}
\bibliography{DKs}

\clearpage

\begin{table}
\begin{center}
\input{tables/props2008-2009.txt}
\end{center}
{\tablenote Data from American National Election Studies 2008-2009 Online Panel.}
\end{table}
\clearpage

\begin{table}
\begin{center}
\input{tables/full2008-2009.txt}
\end{center}
{\tablenote * p<0.05.  Cell entries are linear regression coefficients with associated standard errors in parentheses. Data from American National Election Studies 2008-2009 Online Panel.}
\end{table}
\clearpage

\begin{table}
\begin{center}
\input{tables/anestsdesc.txt}
\end{center}
{\tablenote Cell entries are percentages of respondents providing non-informative responses, by question and survey year. The rightmost column is the average number of non-informative responses across questions for each year. Data from American National Election Studies 2000, 2004, and 2008 time series studies.}
\end{table}
\clearpage

\begin{table}
\begin{center}
\input{tables/fullts.txt}
\end{center}
{\tablenote * p<0.05.  Cell entries are linear regression coefficients with standard errors in parentheses. Data from American National Election Studies 2000, 2004, and 2008 time series studies.}
\end{table}
\clearpage

\begin{table}
\begin{center}
\input{tables/extremity0809.txt}
\end{center}
{\tablenote * p<0.05.  Cell entries are linear regression coefficients with associated standard errors in parentheses. Data from American National Election Studies 2008-2009 Online Panel.}
\end{table}
\clearpage

\begin{table}
\begin{center}
\input{tables/extremityts.txt}
\end{center}
{\tablenote * p<0.05.  Cell entries are linear regression coefficients with associated standard errors in parentheses. Data from American National Election Studies 2000, 2004, and 2008 time series studies.}
\end{table}
\clearpage


\begin{table}
\begin{center}
\input{tables/zallerfeldman.txt}
\end{center}
{\tablenote Data from American National Election Studies 1987 Pilot Study.}
\end{table}
\clearpage






\end{document}